Automatic access of internet content with a camera-enabled cell phone

ABSTRACT

A camera-enabled cell phone that is adapted to image a machine readable code such as a bar code, decode the bar code, send the bar code data over the Internet to a resolution server that will return an associated URL that will ink the camera phone to content on an information server. Thus, by taking a picture of a bar code symbol, the camera phone will automatically retrieve content from the Internet that has been linked to that bar code via the present invention.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority from co-pending U.S.Provisional Application Ser. No. 60/476,496, which was filed on Jun. 6,2003, which is incorporated by reference herein.

BACKGROUND OF THE INVENTION

The present invention relates to wireless access of Internet content, inparticular to the use of a portable camera/cell phone device forscanning bar codes and automatically downloading associated web contentthat is linked to the scanned bar code.

Systems exist in the prior art that allow a user to scan a bar code suchas a product UPC code (or other machine-readable indicia), decode thebar code data, and send the decoded bar code data to an offsite servercomputer, where the server computer looks up a URL associated with thatbar code in a database and sends the retrieved URL back to the user'scomputer. A browser at the user's computer then uses the URL to retrieveweb content associated with the URL. This type of system, for exampledisclosed in U.S. Pat. Nos. 5,978,773 and 6,199,048 (owned by theassignee of the present invention, NeoMedia Technologies, Inc.), allowsa user to automatically link to web content by simply scanning a barcode with a scanner attached to the user's PC. In another type ofsystem, disclosed in U.S. Pat. No. 6,542,933, also owned by NeoMediaTechnologies, Inc., a special bar code known as a PaperClick code isscanned, and the decoded data in parsed into two portions (a server IDand an item ID), with the server ID used to retrieve a URL template thatis sent back to the user's PC, which then assembles a full URL byinserting the item ID from the bar code into the URL template, which isthen used to retrieve the linked web content.

It is desired to be able to utilize this type of automatic web contentretrieval system with portable devices that are not tethered to theuser's PC. Such a portable device would allow a user to automaticallyaccess linked web content at any location, and not just when using hisdesktop PC. However, portable devices that can scan barcodes are notwidely deployed. In addition, it is most convenient for the personscanning a PaperClick code or UPC code to be able to see the resultsimmediately, even if they are not at their computer. A desiredapplication for a portable device under this invention is for a user toenter a Barnes & Noble store, see a book he likes, scan the barcode onthe back, and see what Amazon.com's price is, all without involvinganything that one would normally consider a computer.

SUMMARY OF THE INVENTION

The present invention is a cellular telephone having an embedded orattachable camera and wireless Internet access capabilities, adapted toimage a bar code symbol and retrieve related web content automatically,without the user being tethered to a desktop PC as in the prior art. Theinvention uses a camera phone adapted to perform the desired functionsdescribed herein. By adapting a camera phone in accordance with theinvention, a consumer having such a camera phone can be provided withthe value-added functionality of taking a picture of a bar code on aproduct, and having the camera phone automatically obtain information onthe product, which may be for example prices from a search engine orsponsored web content.

The present invention utilizes a client program that operates a cellphone with a built-in camera and web browser. When the user activatesthe client on the camera phone, it provides a real-time on-screendisplay of what the camera is imaging. When the user presses thetrigger, the camera phone grabs the image, uses an image-based decodealgorithm to locate and decode a barcode, then invokes the built-in webbrowser, pointing it at a resolution server on the Internet with thebarcode value it decoded. The resolution server takes the data sent fromthe camera phone and resolves the data into a URL in one of severalways, depending on the application (to be described later).

Thus, the present invention is a method for accessing content from aninformation server computer on a computer network such as the Internetusing a camera-enabled cell phone. First, an image of a machine-readablecode such as a bar code symbol is captured with the camera-enabled cellphone. The captured bar code image is processed to decode the bar codeinto a file identifier, and a request URL is formed that includes thefile identifier. The request URL is then transmitted to a resolutionserver computer on the computer network. At the resolution servercomputer, an information URL is determined as a function of the requestURL. The information URL is then returned to the camera-enabled cellphone, which in turn transmits the information URL to an informationserver computer designated by the information URL. The informationserver computer receives the information URL and returns content to thecamera-enabled cell phone as a function of the information URL.

The request URL sent by the camera-enabled cell phone may optionallyinclude a device identifier that serves to identify certain operatingcharacteristics of the camera-enabled cell phone, such as the browsercapabilities. In this case, the information URL will be determined as afunction of the device identifier. This allows different informationURLs to be returned for different devices, based on their displaycapabilities, so that each device will retrieve content that isoptimized for display on that particular device.

The information URL may also be determined at the resolution server byextracting the file identifier from the request URL; and then querying adatabase with the file identifier. The resolution server database willhave a plurality of mappings of file identifiers to associatedinformation URLs.

In a further embodiment, the camera-enabled cell phone will capture animage of a machine-readable code with a camera-enabled cell phone andthen process the captured image to decode the machine-readable code intoa file identifier that contains a server ID and an item ID. A requestURL is formed that includes the file identifier. The request URL istransmitted to a predetermined gateway server on the computer network,which analyzes the server ID to determine an associated resolutionserver on the computer network. The gateway server sends the item ID tothe resolution server that is associated with the server ID to obtain aninformation URL as a function of the server ID and the item ID. Theresolution server returns the information URL to the gateway server, andthe gateway server interacts with the camera-enabled cell phone to sendthe information URL to an information server computer designated by theinformation URL and receive content from the information server computeras a function of the information URL.

In this embodiment, the gateway server interacts with the camera-enabledcell phone to send the information URL to an information server computerdesignated by the information URL and receive content from theinformation server computer as a function of the information URL ineither of two ways. First, the gateway server may send the informationURL to the information server computer designated by the informationURL. The gateway server would then receive content from the informationserver computer as a function of the information URL and send thecontent received from the information server to the camera-enabled cellphone. Alternatively, the gateway server may send the information URL tothe camera-enabled cell phone, and the camera-enabled cell phone thensends the information URL to the information server computer designatedby the information URL. The camera-enabled cell phone then receivescontent from the information server computer as a function of theinformation URL.

In order to carry out this invention, the camera-enabled cell phone ofthe present invention includes a housing with a cellular telephonetransceiver, a digital camera module having a camera lens, a displayscreen, and processing circuitry that interoperates with thesecomponents. In particular, the processing circuitry is adapted orprogrammed to cause the cellular telephone transceiver to selectivelycommunicate with a cellular telephone network to place and receivecellular telephone calls, and to communicate with server computers via abrowser program on a global computer network. The processing circuitryis also adapted to capture, via the digital camera module, an image of amachine-readable code, to process the captured image to decode themachine-readable code into a file identifier, to form a request URLcomprising the file identifier and to transmit, via the cellulartelephone transceiver, the request URL to a resolution server computeron the global computer network. The processing circuitry is also adaptedto receive from the resolution server an information URL determined bythe resolution server as a function of the request URL, interoperatewith the browser program to transmit the information URL received fromthe resolution server to an information server computer designated bythe information URL, receive content from the information servercomputer as a function of the information URL, and to display thereceived content on the display screen.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 shows a basic block diagram of the operation of the presentinvention.

FIGS. 2-7 show an embodiment of the lens adapter of the presentinvention.

FIGS. 8-12 show an alternative embodiment of the lens adapter of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 shows a basic block diagram of the operation of the presentinvention. A camera-enabled cell phone 2 (also referred to herein as acamera phone) includes, in one housing, a digital camera module 4, adisplay screen 10, a cellular telephone transceiver 8, processingcircuitry 6, and various user input devices (such as keys, buttons,microphone, touchscreen display) as well as output devices (such as aspeaker) not shown for the purpose of clarity but well known in the artof cell phones. It is noted that although the preferred embodimentdescribed herein utilizes an integrated camera phone (i.e. a cell phonewith a built-in camera), the present invention is also intended tooperate with a cell phone having an attachable camera, e.g. via aninput/output port, as well. Likewise, any device that utilizes thefunctionality of a cellular or wireless phone and an imaging device suchas a digital camera is covered by this invention as well.

A user invokes the appropriate client application on the phone(described below), and then images (takes a picture of), with the camera4, the target bar code symbol 12 or other machine readable code. Theprocessing circuitry 6 decodes the bar code image obtained by the camera4 and generates a request, typically in the form of a request URL havingall or part of the decoded bar code integrated therewith, for sendingout via the cell phone's wireless transceiver 8. This is typically donevia a wireless Internet connection as known in the art.

The request URL is sent wirelessly via a local antenna 14, through thecellular network 16 and the Internet 18, to the destination server,which is referred to as a resolution server 20. The resolution server 20accepts the request URL and parses out the bar code data from it. Theresolution server 20 then looks up the bar code data on a mappingdatabase 27 and retrieves an associated information URL, which is thensent back to the camera phone 2 and handed to a web browser program,which is then redirected by the information URL to the appropriateinformation server 22 on the Internet. The content requested by theinformation URL is sent from the information server 22 to the browser onthe camera phone 2 and displayed thereon for the user. Thus, by invokingthe appropriate client software on the phone 2 as described herein, theuser can image or take a picture of a bar code and have content drivento his display 10 that is associated with that bar code 12.

Various methodologies are known in the art that control how the bar codesymbol is used to retrieve content from the Internet. In the basic casementioned above, there is a simple mapping function carried out by theresolution server 20, which takes the decoded bar code data and looks upa preprogrammed information URL in its database 27. This is shown, forexample, in U.S. Pat. No. 5,978,773, which is incorporated by referenceherein. In an alternative embodiment, the bar code may be parsed into aserver ID and an item ID, as taught in U.S. Pat. No. 6,542,933, which isincorporated by reference herein. Moreover, the content returned to thecamera phone 2 may be specifically tailored for display on the smalldisplay of that device. This may be accomplished by using a methodologyreferred to as device-specific profiled routing, in which the requestURL is assembled by the camera phone based on an additional parameterthat is a device identification code, which designates if that clientdevice is a wireless device that supports, for example, WML content, orXHTML content. By signaling to the resolution server 20 that therequesting device is WAP compliant, then the information URL sent to thecamera phone will redirect its browser to an appropriate web site withWML content (wherein if the same bar code were used to request contentvia a full screen monitor on a desktop, the full HTML page would bereturned). This device based profile routing is fully described in U.S.application Ser. No. 09/821,535, DEVICE-BASED ROUTING FOR WEB CONTENTRETRIEVAL, owned by the assignee of the present invention, thespecification of which is incorporated by reference herein.

In an alternative embodiment, another server computer referred to as agateway server 23 is utilized. The gateway server 23 acts essentially asa proxy server that receives request URLs from the camera phone,processes these requests, and returns either the information URL in aredirect to the camera phone browser or the information server contentdirectly to the browser without requiring a redirect at the cameraphone. In this embodiment, the bar code symbol is a file identifier thatincludes a server ID and an item ID as previously mentioned. The serverID will identify which resolution server will contain the specificmapping of item IDs to information URLs. In this scenario, there will bemore than one (and likely many) resolution servers employed so as theydistribute the database and computing requirements amongst manycomputers. This also has the advantage of allowing multiple parties tohave control over various resolution servers instead of one partycontrolling one large resolution server.

In this embodiment, the request URL contains the server ID and the itemID, and is sent to a gateway server computer 23. Each and every requestURL sent by the camera phone will be sent to the same gateway servercomputer 23 (the address is essentially hard coded into the clientapplication running on the camera phone). The gateway server 23 receivesthe request URL from the camera phone and parses out or extracts theserver ID. The gateway server 23 then analyzes the server ID todetermine which resolution server on the computer network is identifiedthereby (e.g. by reference to a database 25). The gateway server thensends the item ID to the resolution server 20 identified by the serverID. The resolution server 20 receives the item ID and performs a lookupof the item ID to retrieve an associated information URL from memory.That is, the resolution server will prepare and an information URL thatindicates where the requested information may be found. The resolutionserver then returns the information URL to the gateway server 23.

The gateway server then interoperates with the camera phone in one oftwo ways, depending on how the system is configured, to provide theinformation content at the camera phone. In one embodiment, the gatewayserver sends the information URL to the information server, receives thecontent from the information server computer as a function of theinformation URL, and sends the content received from the informationserver to the camera-enabled cell phone.

In an alternative embodiment, the gateway server hands off theinformation URL to the camera phone, the camera phone sends theinformation URL to the information server computer, and then the cameraphone receives the content directly from the information server computeras a function of the information URL.

One problem with camera phones available today is that they are fixedfocus, meaning that the phone is adapted to take pictures at “normal”distances, i.e. several feet out to infinity. Unfortunately, this meansthat pictures taken close (˜3 inches) are badly blurred. To be able todecode the bar code symbol from the captured image data, however, thecamera cell phone must be placed very close to the bar code (e.g.approximately 2-5 inches), otherwise the bars and spaces in the barcodeare too fine for the camera imager, and the bar code cannot be properlydecoded.

As shown in the Figures, a convex lens may be added to the camera with afocal length near the desired distance of about 3 inches. Two types oflens attachments have been developed for use in this invention. Thefirst embodiment (see FIGS. 8-12) has a set of “fingers” that wraparound the cell phone, allowing the lens to be clipped on on so that itis fastened to the phone and over the lens of the camera. In thismanner, the bar codes may be captured with the lens add-on attached, andit may still be removed so the camera phone may be used normally.

A second embodiment is shown in FIGS. 2-7 and is attached to the backcover of the camera. This embodiment allows the lens to be slid up intoplace over the camera lens, and then slid back out of the way. FIG. 2shows a bottom view of the camera phone with this add on attachment inits retracted state, and FIG. 3 shows a bottom view of the camera phonewith this add on attachment in its extended state. FIG. 5 shows a sideview of the camera phone with this add on attachment in its retractedstate, and FIG. 4 shows a side view of the camera phone with this add onattachment in its extended state. FIG. 6 shows a perspective view of thecamera phone with this add on attachment in its retracted state, andFIG. 7 shows a perspective view of the camera phone with this add onattachment in its extended state.

In addition, in order to assist the user to locate the bar code at theoptimal distance from the camera phone, the real-time screen displayincludes index markers on the screen, which basically mark the cornersof a rectangle. The index markers are used to assess how close to holdthe camera phone to the bar code for optimum focus by causing thebarcode to fit within the index markers. Since UPC codes are almostalways a standard size, and since PAPERCLICK proprietary codes are astandard height, fitting the bar code to the rectangle places the barcode at a specific distance from the phone. In practice, the distancemay be varied about +/−one inch without significant degradation, butthis use of index markers aides the user in placing the bar code atabout the center of the field of view for optimal focusing.

1. A method for accessing content from an information server computer ona computer network comprising the steps of: a. capturing an image of amachine-readable code with a camera-enabled cell phone; b. processingthe captured image to decode the machine-readable code into a fileidentifier; c. forming a request URL comprising the file identifier; d.transmitting the request URL to a resolution server computer on thecomputer network; e. determining, at the resolution server computer, aninformation URL as a function of the request URL; f. returning theinformation URL to the camera-enabled cell phone; g. the camera-enabledcell phone transmitting the information URL to an information servercomputer designated by the information URL; h. the information servercomputer receiving the information URL and returning content to thecamera-enabled cell phone as a function of the information URL.
 2. Themethod of claim 1 in which the machine-readable code is a bar codesymbol.
 3. The method of claim 2 in which the bar code symbol isassociated with an object.
 4. The method of claim 3 in which the contentreturned to the camera-enabled cell phone is associated with the object.5. The method of claim 4 in which the bar code symbol is a UPC code. 6.The method of claim 1 in which the computer network is a global computernetwork, and in which the camera-enabled cell phone communicates over acellular network with the global computer network.
 7. The method ofclaim 1 in which the request URL additionally comprises a deviceidentifier.
 8. The method of claim 7 in which the information URL isdetermined as a function of the device identifier.
 9. The method ofclaim 1 in which the step of determining, at the resolution servercomputer, an information URL as a function of the request URL comprisesthe steps of i. extracting the file identifier from the request URL; andii. querying a database with the file identifier; the databasecomprising a plurality of mappings of file identifiers to associatedinformation URLs.
 10. A method for accessing content from an informationserver computer on a computer network comprising the steps of: a.capturing an image of a machine-readable code with a camera-enabled cellphone; b. processing the captured image to decode the machine-readablecode into a file identifier, the file identifier comprising a server IDand an item ID; c. forming a request URL comprising the file identifier;d. transmitting the request URL to a predetermined gateway server on thecomputer network; e. the gateway server analyzing the server ID todetermine an associated resolution server on the computer network; f.the gateway server sending the item ID to the resolution server on thecomputer network that is associated with the server ID, to obtain aninformation URL as a function of the server ID and the item ID; g. theresolution server returning the information URL to the gateway server;h. the gateway server interacting with the camera-enabled cell phone tosend the information URL to an information server computer designated bythe information URL and receive content from the information servercomputer as a function of the information URL.
 11. The method of claim10 wherein the step of the gateway server interacting with thecamera-enabled cell phone to send the information URL to an informationserver computer designated by the information URL and receive contentfrom the information server computer as a function of the informationURL comprises the steps of: i. the gateway server sending theinformation URL to the information server computer designated by theinformation URL; ii. the gateway server receiving content from theinformation server computer as a function of the information URL; iii.the gateway server sending the content received from the informationserver to the camera-enabled cell phone.
 12. The method of claim 10wherein the step of the gateway server interacting with thecamera-enabled cell phone to send the information URL to an informationserver computer designated by the information URL and receive contentfrom the information server computer as a function of the informationURL comprises the steps of: i. the gateway server sending theinformation URL to the camera-enabled cell phone; ii. the camera-enabledcell phone sending the information URL to the information servercomputer designated by the information URL; iii. the camera-enabled cellphone receiving content from the information server computer as afunction of the information URL.
 13. A method for accessing content froman information server computer on a computer network comprising thesteps of: a. initiating a client application on a camera cell phone, thecamera cell phone having wireless access to a global computer network,the client application adapted to interoperate with a browser program onthe camera cell phone for accessing the global computer network; b.taking a picture of a bar code symbol associated with an object with thecamera cell phone; c. processing the captured image to decode the barcode and determine the bar code type and the bar code value; d. forminga request URL comprising the bar code type, the bar code value, and adevice identifier that identifies display capabilities of the browserprogram; e. transmitting the request URL via the cell phone network to apredetermined resolution server computer on the global computer network;f. extracting, at the resolution server computer, the bar code type, thebar code value, and the device identifier; g. using the bar code type,the bar code value, and the device identifier to lookup an informationURL in a database at the information server computer; h. returning theinformation URL to the browser program of the camera cell phone; i. thebrowser program of the camera cell phone transmitting the informationURL to an information server computer designated by the information URL;j. the information server computer receiving the information URL andreturning content to the camera cell phone as a function of theinformation URL.
 14. A system for accessing content from an informationserver computer on a computer network comprising: A. a camera-enabledcell phone comprising: a. a cellular telephone transceiver; b. a digitalcamera module; c. a display screen; and d. processing circuitry adaptedto: i. cause the cellular telephone transceiver to selectivelycommunicate with a cellular telephone network to place and receivecellular telephone calls, and to communicate with server computers via abrowser program on a global computer network; ii. capture, via thedigital camera module, an image of a machine-readable code; iii. processthe captured image to decode the machine-readable code into a fileidentifier; iv. form a request URL comprising the file identifier; v.transmit, via the cellular telephone transceiver, the request URL to aresolution server computer on the global computer network; B. aresolution server computer interconnected to the global computernetwork, the resolution server computer comprising processing circuitryadapted to a. determine an information URL as a function of the requestURL received from the camera-enabled cell phone; and b. return theinformation URL to the camera-enabled cell phone; wherein the processingcircuitry of the camera-enabled cell phone is further adapted tointeroperate with the browser program to transmit the information URLreceived from the resolution server to an information server computerdesignated by the information URL; to receive content from theinformation server computer as a function of the information URL; and todisplay the received content on the display screen.
 15. The system ofclaim 14 in which the machine-readable code is a bar code symbol. 16.The system of claim 15 in which the bar code symbol is associated withan object.
 17. The system of claim 15 in which the content returned tothe camera-enabled cell phone is associated with the object.
 18. Thesystem of claim 17 in which the bar code symbol is a UPC code.
 19. Thesystem of claim 14 in which the request URL additionally comprises adevice identifier.
 20. The system of claim 19 in which the informationURL is determined as a function of the device identifier.
 21. The systemof claim 14 in which the resolution server processing circuitry isadapted to determine an information URL as a function of the request URLreceived from the camera-enabled cell phone by i. extracting the fileidentifier from the request URL; and ii. querying a database with thefile identifier; the database comprising a plurality of mappings of fileidentifiers to associated information URLs.
 22. A camera-enabled cellphone for accessing content from an information server computer on acomputer network comprising: a. a cellular telephone transceiver; b. adigital camera module; c. a display screen; and d. processing circuitryadapted to: i. cause the cellular telephone transceiver to selectivelycommunicate with a cellular telephone network to place and receivecellular telephone calls, and to communicate with server computers via abrowser program on a global computer network; ii. capture, via thedigital camera module, an image of a machine-readable code; iii. processthe captured image to decode the machine-readable code into a fileidentifier; iv. form a request URL comprising the file identifier; v.transmit, via the cellular telephone transceiver, the request URL to aresolution server computer on the global computer network; vi. receivefrom the resolution server an information URL determined by theresolution server as a function of the request URL; vii. interoperatewith the browser program to transmit the information URL received fromthe resolution server to an information server computer designated bythe information URL; viii. receive content from the information servercomputer as a function of the information URL; and ix. display thereceived content on the display screen.
 23. A camera-enabled cell phonefor accessing content from an information server computer on a computernetwork comprising a housing comprising: a. a cellular telephonetransceiver; b. a digital camera module comprising a camera lens; c. adisplay screen; and d. processing circuitry adapted to: i. cause thecellular telephone transceiver to selectively communicate with acellular telephone network to place and receive cellular telephonecalls, and to communicate with server computers via a browser program ona global computer network; ii. capture, via the digital camera module,an image of a machine-readable code; iii. process the captured image todecode the machine-readable code into a file identifier; iv. form arequest URL comprising the file identifier; v. transmit, via thecellular telephone transceiver, the request URL to a gateway servercomputer on the global computer network; vi. receive from the gatewayserver computer an information URL determined by an associatedresolution server as a function of the request URL; vii. interoperatewith the browser program to transmit the information URL received fromthe gateway server to an information server computer designated by theinformation URL; viii. receive content from the information servercomputer as a function of the information URL; and ix. display thereceived content on the display screen.
 24. The camera-enabled cellphone of claim 23 further comprising an external lens module adapted tofit over the housing so as to align with the camera lens and providemagnification of the machine readable code sufficient to allowsuccessful decoding of the captured image of the machine readable code.25. The camera-enabled cell phone of claim 24 wherein the external lensmodule clips onto the housing.
 26. The camera-enabled cell phone ofclaim 24 wherein the external lens module is slidingly engaged with thehousing.
 27. A camera-enabled cell phone for accessing content from aninformation server computer on a computer network comprising a housingcomprising: a. a cellular telephone transceiver; b. a digital cameramodule comprising a camera lens; c. a display screen; and d. processingcircuitry adapted to: i. cause the cellular telephone transceiver toselectively communicate with a cellular telephone network to place andreceive cellular telephone calls, and to communicate with servercomputers via a browser program on a global computer network; ii.capture, via the digital camera module, an image of a machine-readablecode; iii. process the captured image to decode the machine-readablecode into a file identifier; iv. form a request URL comprising the fileidentifier; v. transmit, via the cellular telephone transceiver, therequest URL to a gateway server computer on the global computer network;vi. receive from the gateway server computer an information URLdetermined by an associated resolution server as a function of therequest URL; vii. interoperate with the browser program to transmit theinformation URL received from the gateway server to an informationserver computer designated by the information URL; viii. receive contentfrom the information server computer via the gateway server computer asa function of the request URL; and ix. display the received content onthe display screen.
 28. The camera-enabled cell phone of claim 27further comprising an external lens module adapted to fit over thehousing so as to align with the camera lens and provide magnification ofthe machine readable code sufficient to allow successful decoding of thecaptured image of the machine readable code.
 29. The camera-enabled cellphone of claim 28 wherein the external lens module clips onto thehousing.
 30. The camera-enabled cell phone of claim 28 wherein theexternal lens module is slidingly engaged with the housing.